In the context of Federated Reinforcement Learning (FRL), applying Proximal Policy Optimization (PPO) faces challenges related to the update order of its actor and critic due to the aggregation step occurring between successive iterations. In particular, when local actors are updated based on local critic estimations, the algorithm becomes vulnerable to data heterogeneity. As a result, the conventional update order in PPO (critic first, then actor) may cause heterogeneous gradient directions among clients, hindering convergence to a globally optimal policy. To address this issue, we propose FedRAC, which reverses the update order (actor first, then critic) to eliminate the divergence of critics from different clients. Theoretical analysis shows that the convergence bound of FedRAC is immune to data heterogeneity under mild conditions, i.e., bounded level of heterogeneity and accurate policy evaluation. Empirical results indicate that the proposed algorithm obtains higher cumulative rewards and converges more rapidly in five experiments, including three classical RL environments and a highly heterogeneous autonomous driving scenario using the SUMO traffic simulator.

Script knowledge consists of detailed information on everyday activities. Such information is often taken for granted in text and needs to be inferred by readers. Therefore, script knowledge is a central component to language comprehension. Previous work on representing scripts is mostly based on extensive manual work or limited to scenarios that can be found with sufficient redundancy in large corpora. We introduce the task of scenario detection, in which we identify references to scripts. In this task, we address a wide range of different scripts (200 scenarios) and we attempt to identify all references to them in a collection of narrative texts. We present a first benchmark data set and a baseline model that tackles scenario detection using techniques from topic segmentation and text classification.

A common problem for companies with strong business growth is that it is hard to find enough experienced staff to support expansion needs. This problem is particular pronounced for operations planners and controllers who must be very highly knowledgeable and experienced with the business domain. This article is a case study of how one of the largest travel agencies in Hong Kong alleviated this problem by using AI to support decision-making and problem-solving so that their planners and controllers can work more effectively and efficiently to sustain business growth while maintaining consistent quality of service. AI is used in a mission critical fleet management system (FMS) that supports the scheduling and management of a fleet of luxury limousines for business travelers. The AI problem was modeled as a constraint satisfaction problem (CSP). The use of AI enabled the travel agency to sign up additional hotel partners, handle more orders and expand their fleet with their existing team of planners and controllers. Using modern web 2.0 architecture and proven AI technology, we were able to achieve low-risk implementation and deployment success with concrete and measurable business benefits.